View Issue Details

IDProjectCategoryView StatusLast Update
0001246T99X171.00 SKB EagleVOCpublic2021-06-23 09:57
Reporter(ALTech) Younkwang Jung Assigned To(SW) Jim Chen Due Date
PriorityimmediateSeveritys2-severeReproducibilityhave not tried
Status closedResolutionfixed 
Summary0001246: [Smart3][BTF][VoC] Booting fail issue after upgrade to Android 10
DescriptionHi Jacky

Booting fail issue occurred in 116 STBs.
There was an issue during the update from v15.520.8 to v15.523.6/v15.524.4 ( from Android 9 to 10).
If this issue occurs, STB needs to be replaced. (user can't proceed any further because it stopped at bootloader. )

The bootloader error log is as follows.
==================================
[05/20, 13:49:08] AVB2 verify with default kpub
[05/20, 13:49:10] avb_slot_verify.c:343: ERROR: recovery: Hash of data does not match digest in descriptor.
[05/20, 13:49:10] avb verification: locked = 1, result = 3
==================================
please check attached serial log ( booting_fail_serial_log.zip )
An error is occurring while checking the recovery area.

I did flash dump at 3 STBs. the results are as follows. ( boot_fail_dump_2E_0F.zip / boot_fail_dump_12_E6.zip / boot_fail_dump_A5_5D.zip )
bootloader.dump ==> 523.6 version ( android 10 )
dtbo.dump ==> 520.8 version ( android 9 )
recovery.dump ==> 523.6 version ( android 10 )
vbmeta.dump ==> 520.8 version ( android 9 )

The total change from v15.520.8 to v15.523.6 has not been made. I don't know the current cause of this situation.
Please check the information on the possibility of this happening.

For your information, I wrote vbmeta(v15.523.6) on the booting failed board, and it worked normally.
(Upgraded after entering the revocery mode)

OTA service has been stopped due to this issue.
I ask for your prompt support on this issue.

Thank you
YK.Jung
TagsNo tags attached.
Attach Tags

Users monitoring this issue

User List (ALTech) SY Yoon , (SW) Brent Choi , (SW) Jacky Chiang , (SW) Jim Chen , (SW) Kerwin Chen

Activities

(ALTech) Younkwang Jung

2021-05-25 13:44

developer  

boot_fail_dump_2E_0F.zip (25,419,371 bytes)
boot_fail_dump_12_E6.zip (25,419,371 bytes)
boot_fail_dump_A5_5D.zip (25,419,371 bytes)

(ALTech) SY Yoon

2021-05-26 09:21

developer   ~0007231

HI Mr. Jim Chen,

Could you update your progress on this issue review ?
Can you explain current status with AMLOGIC ?

By the way, I asked Mr. Jacky Chiang to update Foxconn's progress every 5:00 PM in your time.
I should update our review result and action plan to SKB everyday.
Please keep in mind.

(SW) Jim Chen

2021-05-26 22:40

developer   ~0007262

Hi, Mr. Yoon,

Mr. YK and AML has digged this issue for a while.
I can only learn their knowhow from Jira,
It's hard for me to have something new to update.

Since the solution currently is to "repair" when the issue happen,
I'll try to find "why" the issue happen.
However with the probability that is 0.1~0.3%,"repair" way is much more efficient.

The plan is to make an auto test environment,
to see if there's any clue on the console when issue happen,
and will also be usable for verifying the solutions from AML.

Here's the items need to be solved:
1. study the linux based burning tool. (ok)
2. linux based burning tool not stable.
3. make special version of A9/A10 that will (ota/burn) to each other
4. OTA should through network since USB port is occupied by A-A cable

(ALTech) SY Yoon

2021-05-27 07:55

developer   ~0007263

Hi Jim Chen,

Do you have any experience to update OS version for your any product ?
If you have, please share your experience to us.

(SW) Jim Chen

2021-05-27 09:48

developer   ~0007268

Hi Mr. Yoon,

Most update are just flash(nand/nor/emmc) partition update with check algorithm.
The special part of this issue is that we use 2-steps update, that is:
1. update bootloader, recovery (maybe more) partition, and reboot again to enter "new recovery"
2. new recovery will handle the rest of the partition update.

my rough guess is,
reboot of step1 is too fast so that some data doesn't sync well in emmc
so after reboot we get into trouble with those check algorithm
however there's no evidence for above,
and If we add some delay before step1 reboot, we need an auto test environment to verify it.

(SW) Jim Chen

2021-05-28 09:54

developer   ~0007287

to simplify the auto test environment, here's the changed plan
1. make 2 special Android10 FW
2. force enable the 2-steps update
3. assign different USB FW upgrade path
4. after bootup, automatically reboot into recovery to update.


and here update some of my study, please check with attached
the red part is step1, we can see 3 partitions are update:
 bootloader.img
 dt.img
 recovery.img

if we want to dealy before reboot, it could put in set_bootloader_env
it's a c-code func that we can add some flush() and sleep()
updater-script.PNG (51,208 bytes)   
updater-script.PNG (51,208 bytes)   

(ALTech) Younkwang Jung

2021-06-23 09:57

developer   ~0007531

Hi Jim Chen

This issue has been resolved.

Thank you for your support.
YK.Jung

Issue History

Date Modified Username Field Change
2021-05-25 13:44 (ALTech) Younkwang Jung New Issue
2021-05-25 13:44 (ALTech) Younkwang Jung Status new => assigned
2021-05-25 13:44 (ALTech) Younkwang Jung Assigned To => (SW) Jacky Chiang
2021-05-25 13:44 (ALTech) Younkwang Jung File Added: booting_fail_serial_log.zip
2021-05-25 13:44 (ALTech) Younkwang Jung File Added: boot_fail_dump_2E_0F.zip
2021-05-25 13:44 (ALTech) Younkwang Jung File Added: boot_fail_dump_12_E6.zip
2021-05-25 13:44 (ALTech) Younkwang Jung File Added: boot_fail_dump_A5_5D.zip
2021-05-25 13:44 (ALTech) Younkwang Jung Issue Monitored: (ALTech) SY Yoon
2021-05-25 13:44 (ALTech) Younkwang Jung Issue Monitored: (SW) Brent Choi
2021-05-25 13:44 (ALTech) Younkwang Jung Issue Monitored: (SW) Kerwin Chen
2021-05-25 14:30 (SW) Jacky Chiang Assigned To (SW) Jacky Chiang => (SW) Jim Chen
2021-05-26 09:16 (ALTech) SY Yoon Issue Monitored: (SW) Jacky Chiang
2021-05-26 09:18 (ALTech) SY Yoon Issue Monitored: (SW) Jim Chen
2021-05-26 09:21 (ALTech) SY Yoon Note Added: 0007231
2021-05-26 22:40 (SW) Jim Chen Note Added: 0007262
2021-05-27 07:55 (ALTech) SY Yoon Note Added: 0007263
2021-05-27 09:48 (SW) Jim Chen Note Added: 0007268
2021-05-28 09:54 (SW) Jim Chen File Added: updater-script.PNG
2021-05-28 09:54 (SW) Jim Chen Note Added: 0007287
2021-06-23 09:57 (ALTech) Younkwang Jung Status assigned => closed
2021-06-23 09:57 (ALTech) Younkwang Jung Resolution open => fixed
2021-06-23 09:57 (ALTech) Younkwang Jung Note Added: 0007531